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This result-based paper presents the best results of both fitted BPNN-NAR 
and BPNN-NARMA on MCCI Aggregate dataset with respect to different 
error measures. This section discusses on the results in terms of the 
performance of the fitted forecasting models by each set of input lags and 
error lags used, the performance of the fitted forecasting models by the 
different hidden nodes used, the performance of the fitted forecasting models 
when combining both inputs and hidden nodes, the consistency of error 
measures used for the fitted forecasting models, as well as the overall best 
fitted forecasting models for Malaysian aggregate cost indices dataset. 
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1. INTRODUCTION 

According to [1], input lags alone may not be adequate to precisely rough the underlying generation 
process. He added that one lag is not able to capture the underlying relationship or not sufficient to model a 
process, thus there is a need of adequate number of hidden neurons for optimal network performance. Both 
inputs and hidden nodes can significantly affect the learning of a neural network when forecasting a time 
series. This research is interested in accuracy of the models instead of parsimonious time series models, as 
suggested by [2]. For time series, inputs also include lags. As the input dimensionality increases, model 
complexity increases and learning becomes more difficult, leading to poor convergence. The challenge here 
is to select or find the best combinations of input lags with adequate number of hidden nodes that will lead to 
a superior model, specifically to a particular dataset of a certain scenario. If the training set is sufficiently 
large, ANN will generalise accurately and will produce accurate outputs for inputs not in the training set. 
When neural networks are properly trained, they can generalise and extrapolate additional details of the 
function mapping the inputs to outputs. Nevertheless, no prior assumptions of dependent and independent 
variables need to be made since the neural network is trained on observed data [3], [4]. This infers neural 
systems are ordinarily used as "black-box" devices that are no earlier information about the method was 
assumed; the goal was to develop a process model based only on observations of its input-output behaviour 
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[5-7]. By definition, the black-box is a modelling technique that constructs the model just based on the 
information gained from the framework, and it doesn't depend on upon other information about the system 
[8]. No early presumptions about the model structure are made, and rather, the modelling technique’s concern 
is to make a generic model that maps the information yield relationship of the dataset [9, 10]. Revelation 
models are known to be viable and versatile, as the assistant information about the system may not be 
promptly accessible. Erom this investigation, one shortcoming lies in the way that that the model complexity 
expands [7] Also, another favorable position of the black-box identification is that it can demonstrate flow 
that are inadequately caught by mathematical models [12], [13]. 


2. METHODOLOGY 

In this research, Malaysian Aggregate monthly data from January 1980 to December 2013 were 
adapted. These datasets are secondary data collected from three different sources, UKAS, from Malaysian 
Prime Minister’s Office, Malaysian CIDB, and Malaysian Statistical Department. In this research, only 
Malaysian central region aggregate price indices were adapted which comprise of 3.9% outliers. The total 
N=408 (12 months x 34 years) from January 1980 to December 2013 (base 1980=100). The mean of 
Malaysian Aggregate is 113.7731. Based on the Jarque-Bera test for normality, the variable aggregate (J- 
B=0.873, p=0.000) is highly significant at 99% confidence interval. 

Eirst and foremost, this research discusses the performance of the fitted forecasting models on 
Malaysian aggregate cost indices data by each set of input lags and error lags used. Table 1 shows 
performance results of fitted forecasting models of two-layer tansig/linear transfer functions on Malaysian 
aggregate cost indices data. Eor comparison purposes, this research used input lags from 5 to 40 and hidden 
nodes from 5 to 45 for BPNN-NAR model, while for the BPNN-NARMA model, this research used the error 
lags ranging from 5 to 40, together with input lags from 5 to 40 and hidden nodes from 5 to 45. The most 
common approach to decide this property is experimentation or trial-and-error [14-16], albeit different 
strategies (rule-of-thumb) and algorithms (pruning and growing) are additionally accessible [17]-[19]. It is to 
be noticed that BPNN-NAR is a feedforward neural network type model, while BPNN-NARMA is a 
recurrent neural network type model [18]. This exploration reports the error measures on the test dataset, 
which is the most vital characteristic, reflecting ANN’s generalisation ability [19], [20]. Table 2 shows the 
data partitioning for network pre-processing. Eigure 1 shows the flowchart of the BPNN-NAR and BPNN- 
NARMA. 


Table 1. Combinations of Input Lags, Error Lags and Percentage of Outliers for BPNN - NAR and BPNN - 

NARMA Models 





ANN Models 



No. Data Type Notation 

Outliers 

NAR 

NARMA 

Hidden 



Input Lags 

Input Lags 

Error Lags 

Nodes 



5, 

5, 

5, 

5, 



10, 

10, 

10, 

10, 



15, 

15, 

15, 

15, 



20, 

20, 

20, 

20, 
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35, 

35, 
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35, 



40 

40 

40 

40, 
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Table 2. Size of Data Partitioning for Training, Validation and Testing Sets for Each Data Used in This 



Research 




No. Data Type 

Total Sample 


In-sample Data 


Out-of-sample Data 

Size (N) 

Training (70%) Validation (15%) 

Testing (15%) 

j Monthly Malaysian Aggregate 

408 

286 


61 

61 

Price Indices Data 
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Figure 1. Flowchart of Backpropagation Neural Network NAR and NARMA Mechanism 


3. RESEARCH FINDINGS 

Figure 2 shows the performance of BPNN-NAR with respect to input lags and hidden nodes on 
Malaysian aggregate cost indices data based on RMSE. Whereas, Figure 3 shows the performance of BPNN- 
NARMA with respect to input lags and hidden nodes on Malaysian aggregate cost indices data based on 
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RMSE. Erom the radar diagrams of both figures, this research can clearly see that overall of the lines of 
BPNN-NAR are more towards the center of zero values, meaning zero error compared to BPNN-NARMA. 
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Eigure 2. Performance of BPNN-NAR with respect to input lags and hidden nodes on Malaysian Aggregate 

cost indices data based on RMSE 
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Figure 3. Performance of BPNN-NARMA with respect to input lags, error lags and hidden nodes on 
Malaysian aggregate cost indices data based on RMSE 


Based on input and error lags 5, the optimal number of hidden nodes was 20. Based on input and 
error lags 10, the optimal number of hidden nodes was 20. Based on input and error lags 15, the optimal 
number of hidden nodes was 15. Based on input and error lags 20, the optimal number of hidden nodes was 
20. Based on input and error lags 25, the optimal number of hidden nodes was 25. Based on input and error 
lags 30, the optimal number of hidden nodes was 30. 

Based on these results, BPNN-NARMA model performed worse than the BPNN-NAR model. By 
right, BPNN-NARMA should perform better than BPNN-NAR [21]-[23]. This is perhaps due to the outliers 
in the data. In most cases, as the number of input and error lags increased, the errors are also increased. 

Furthermore, this research discusses on the performance of the fitted forecasting models by different 
hidden nodes used. From Table 1, over the input and error lags, if the network is assigned with fewer nodes 
or huge load of nodes, the performance of the network will get worse. This happens due to the network 
overfitting problem [24]-[28]. Therefore, an adequate number of hidden nodes is important in neural network 
modelling [29]-[31]. It can be concluded that if the network is given an adequate number of hidden nodes, the 
performance of the network will get better and model errors will be reduced, and the chance of getting a more 
accurate model can be achieved. 

In terms of the consistency of error measures used for the fitted forecasting models, this research 
concludes that RMSE performed consistently. This is supported by recent works in the forecasting arena such 
as [32-36]. Moreover, in most cases, many researchers used RMSE to evaluate their forecasting models such 
as [37, 38]. Therefore, RMSE will be the used to assess the performance of the models throughout this 
research. The best results for this section based on RMSE can be summarized in Table 2. 

Lastly, it can be concluded that the best fitted model for Malaysian aggregate cost indices data was 
the two-layer tansig/linear transfer functions BPNN-NAR model with 10-20 configurations (RMSE=0.584, 
MSPE=0.341, MAPE=55.378, MAD=0.429, and GRMSE=0.782), can be seen in Table 3. 
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Table 3. Best Results of Ordinary BPNN-NAR and BPNN-NARMA Models on Malaysian Aggregate Cost 

Indices Data based on Different Lags 


Input 

Lags 

Error 

Lags 

Hidden 

Nodes 

BPNN-NAR 

RMSE 

BPNN- NARMA 

5 

5 

20 

0.857 

1.036 

10 

10 

20 

0.584 

0.626 

15 

15 

15 

0.744 

0.808 

20 

20 

20 

0.766 

0.850 

25 

25 

25 

0.807 

0.892 

30 

30 

30 

0.834 

0.931 

35 

35 

35 

0.843 

0.950 

40 

40 

45 

0.882 

1.003 


4. CONCLUSION 

The investigation of BPNN-NAR on MCCI data demonstrates that higher input lags implies higher 
RMSE. Correspondingly, the higher or the lesser hidden nodes to the input lags, the higher the RMSE, and 
the higher the input lags, the higher the RMSE. 

Despite the fact that NARMA is superior to NAR, when there are outliers in the dataset, the 
NARMA display appeared to separate, and the NAR demonstrate outflanked the NARMA show. Here, this 
examination proposes that the NAR model ought not be amplified or gone before by NARMA demonstrate 
when the dataset comprises of outliers. 

The NARMA shows performed more terrible than NAR when there exist outliers in the time 
arrangement datasets. NARMA model should perform better contrasted with NARMA demonstrate. This can 
be demonstrated by the aftereffects of both models when fitting Malaysian sand cost indices data, which don't 
comprise of outliers issue. Erom the results, it is clear that that there is a need to modify NAR and NARMA 
models so that they may handle outliers issue effectively, or else the expansion of NAR model to NARMA 
model ought not be continued since the NARMA model will tend deliver bigger errors, and the outcomes are 
not solid for further utilize. In the future, the results will be compared to the existing works of [39], [40]. 
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